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ABSTRACT 

Achievement test scores and measures of district and 
pupil characteristics were analyzed for all California school 
districts. Achievement test scores were found to be highly correlated 
among each other. The best single predictor of achievement scores vas 
a measure of family poverty. Scholastic aptitude scores were not used 
as predictors. Regression equations which were developed to predict 
achievement scores on the basis of district characteristics accounted 
for about half of the variance in achievement scores among districts. 
(Author) 
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ABSTRACT 



Achievement test scores and measures of district and pupil character- 
istics were analyzed for all California school districts* Achievement test 
scores were found to be highly correlated among each other. The best single 
predictor of achievement scores was a measure of family poverty. Scholas- 
tic aptitude scores were not used as predictors. Regression equations 
developed to predict achievement scoros on the basis of district character- 
istics accounted for about half of the variance in achievement scores 
among districts. 
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INTRODUCTION 

The purposes of this technical supplement axe (1) to describe the pro- 
cedures used in computing the converted and predicted scores presented in 
the report , California State Testing Program , 1970-71 , Profiles of Sc hool 
District Performance y and (2) to present more complete statistical details 
resulting from the analyses performed* 

The purposes of the analyses were (1) to gain a better understanding 
of the relationships among achievement test scores and measures of district 
characteristics and resources and (2) to apply this knowledge to the tech- 
nique of predicting test scores for a district by using those predicted 
scores as a basis for setting expectancies. This method of evaluating test 
results in the light of other information about a school district was first 
used in the publication, California State Testing Program , 1969-70 , a District 
by District Analysis of Test Scores and Other School Factors. Changes have 
been made in the report for 1970-71 in order to make it more imderstandable 
and useful. 

DATA TRANSFORMATIONS 
District median test scores and other factors were first converted to 
statewide percentile ranks, and then to statewide normalized z-scores. 
These z-scores indicate the number of standard deviation units above or below 
the mean to which each percentile rank would correspond in a normal distri- 
bution. The z-scores were used to calculate a correlation matrix for each 
type of district. When data were missing, the correlations among the avail- 
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able data were used as the best estimate of the corresponding correlation 
coefficients. This procedure yielded results which must be considered ap- 
proxj'^i te, since it assumes that the missing data are not different in kind 
from the available data. The missing information on any one variable gener- 
ally involved data from less than 3 percent of the districts • All subse- 
quent analyses were based on the normalized z-scores, 

CORRELATIONAL ANALYSIS 

E] even achievement test scores and 22 district descriptors were avail- 
able for all unified districts*- The number of variables was correspondingly 
lower for elementary and high school districts. Tables 1-9 present the cor- 
relation coefficients among these scores and descriptors for \mified, elemen- 
tary, and high school districts, 

A correlation coefficient is a measiire of the relationship between two 
variables. This statistic can assume values ranging from -1,0 through zero 
to +1,0, A positive correlation coefficient indicates that as the value of 
one variable increases, the value of the other increases; that is, high 
values on one tend tr> be accompanied by high values on the other. For ex- 
ample, from Table 3t it can be seen that scholastic ability scores for 6th- 
grade pupils correlated ,92 with reading scores in the 6th grade. This 
indicates a strong relationship between these variables, A correlation of 
zero indicates that there is no tendency for high or low scores on one 
variable to occur, or to correlate with high or low scores on the other, A 
further example from Table 3 shows that average class size correlated ,00 with 
12th-grade language, indicating there was essentially no relationship or 
agreement between the ordering of these two measures, A negative correlation 
coefficient indicates that as the value of one variable increases, the value 
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of the other variable decreases. Again from Table 3 it can be seen that 
Index of Family Poverty correlated -.60 with 12thrgrade mathematics scores. 
This indicates that higher levels of poverty tend to be associated with 
lower mathematics achievement scores. 

Unified School Districts 

Tables 1, 2, and 3 present intercorrelation matrices for the measures 
for all California \inified school districts. 

The tables provide for an easy examination of the relationships among 
the 11 achievement test scores (output measures) (Table l), the 22 input 
measures (Table 2), and the correlations between the test scores and the pre- 
dictor measures (Table 3) • 

By examining; Table 1, one can see that the median test scores are higjily 
intercorrelated. The correlation coefficients range from .25 to .90 with 
many in the .70's and •80's. This is not \inexpected since districts tend 
to be fairly homogeneous. Factors affecting pupil achievement in one grade 
level or ciirricular area may be expected to have a similar impact in other 
areas. At a given grade level the measures would be expected to correlate 
highly because they measure similar cognitive skills of the same children. 

The coefficients presented in Table 2 for the predictor or input vari- 
ables range from -.69 to +.86* Most of the high positive coefficients are 
cunong measures which one would expect to be highly related; e.g., general 
fund tax rate and general purpose tax rate (•86), class size in grades one 
through three and pupil-teacher ratio in grades four through eight (.58), 
median teacher salary and maximum teacher salary (.75) 1 and corresponding 
measures of pupil ability at grade 6 and 12 (.79) • These positive correla- 
tions indicate that high values in one member of the pair tend to be associ- 
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ated with high values in the other member of the pair, and low values tend 
to be associated with low values. 

The highest negative correlations Table 2 are between measures of 
scholastic ability and the social*- financial descriptors: percent total mi- 
nority enrollment (-.61, -.61); percent Spanish surnaraed (-.6l, -.51); and 
index of family poverty (-.691 -.67) • In other words, the larger the percent 
of minority enrollment or the higher the index of family poverty, the lower 
the values of scholastic ability. Other negative relationships were apparent 
between assessed valuation and the variables of: average daily attendance; 
class size (-.61); puprl-teacher ratio (-.52); and general fund tax rate 
(-.61f). 

Ebqjenditures for instruction per pupil was negatively correlated with 
average class size (-.53) and pupil-teacher ratio (-.^5). Expenditures for 
instruction per pupil was not related to percent total minority pupils (-.03), 
but was positively related to percentage of Black students (.11) and negatively 
related to percentage of Spanish surnamed students (-.24), indicating a 
very slight tendency for more resources to be available for school districts 
with a high percentage of Black students; and a tendency to have fewer re- 
sources for school districts with larger Spanish surnamed populations. 

Table 3 presents correlations observed between the 22 predictor meastires 
and the 11 test scores. Scholastic ability scores eire very closely related 
to achievement test scores. The coefficients range from . 44 to .92 with many 
in the 70 's and 80's. This is to be expected since scholastic ability tests 
involve the use of basic skills which also underlie achievement in many areas. 
They also tend to be similar to achievement tests because of foxTnat and other 
constraints imposed by the nature of multiple-choice tests. 



Several hypotheses can be offered to account for these high correlations 
among achievement scores and scholastic ability measures. First, it is lo- 
gical to expect pupils with higher scholastic abilities to do better on achieve- 
ment tests than pupils with lower scholastic abilitils. Also, the scholastic 
ability tests used here and the achievement tests are quite similar in many, 
ways. They not only are alike in format and design, but the test content in 
terms of the skills and concepts which the tests measure are also similar. 
In some ways, the scholastic ability tests may be considered to be achievement 
tests of general academic skills as well as measures of application and reason- 
ing with those skills. The scholastic ability tests used here sire not culture- 
free. These tests use the concepts and experiences common to a culture to 
estimate the relative degree to which pupils may be expected to profit from 
instruction. This means that a child with a backgromid of rich educational 
experiences is likely to score well on ability tests as well as on achievement 
tests. It also means that a child with a less rich experience, especially if 
he has a different language background, is likely to do poorly on both ability 
tests and achievement tests for the same reason. 

The next highest coefficients are negative ones. Percent of minority 
enrollment and index of family poverty are inversely related to school achieve- 
ment. Most of the coefficients range in the -.^O's and -.60's, which indicate 
relatively strong inverse relationships among these input measures and school 
achievement. 

Other vEiriables which correlated moderately positively with achievement 
were teacher salaries and size of district. Expenditures for instruction and 
ratio of nonteaching certificated personnel to full-time equivalent teachers 
bore very low, positive relationships with achievement. 
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other variables which were slightly negatively related to achievement 
were pupil mobility and to a lesser degree, rate of staff turnover, class 
size, and pupil-teacher ratio. Tax rates and levels of assessed valuation 
per pupil were uzirelated to school achievement. 

Elementary School Districts 

Tables ^, ^^and 6 present the intercorrelation matrices for elementary 
school districts. Relationships among the test scores are presented in Table 
k. Relationships are similar to those for unified school districts, i.e., all 
are positive and generally. moderate to high, indicating that achievement in 
one area tends to be associated with achievement in another area. 

Relationships among predictor variables, as presented in Tablo 5, were 
also similar to those found for unified districts. 

The relationships between the test variables and the predictor variables, 
as shown on Table 6, are also so similar to those for unified districts that 
it is unnecessary to describe them. ITie positive relationship between non- 
teaching certificated personnel and the achievement test scores is somewhat 
lower for elementary districts, as are the negative coefficients between 
class size and school achievement and between pupil-teacher ratio and school 
achievement. 

High School Districts 

Tables 7, 8, and 9 present the intercorrelation matrices for high school 
districts. As displayed in Table 7i the relationships among the four test 
variables su^e all quite high. The correlations among the predictor variables 
are presented on Table 8. The pattern of relationships is very similar to that 
observed for the unified and elementary districts. Slight differences in 
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degree of relationship appear between certain variables;. e.g., the positive 
relationship between expenditures for instruction and teachers' salaries is 
weaker for high school districts than for unified or elementary districts. 
From Table 9 it can be seen that the pattern of relationships between 
• the test variables and the predictor vsiriables is simileo* to that observed for 

other types of districts with one exception; for elementary and unified school 
districts a low, positive relationship was found between test scores and non-^ 
teaching certificated personnel, whereas for high school districts the relation- 
ship is low and negative. 

EEGEESSION ANAJ.YSIS 

Purpose ' 

It is desirable to have a single number which summarizes the effective- 
ness of a district on particular achievement dimensions. A raw test scor^ is 
less accurate than desired for this purpose because it does not take into ac- 
count differences in available renources among districts. This section de- 
scribes an attempt to meet this need by preparing a prediction for a district 
on the basis of the achievement of districts with similar resources. A devi- 
ation from this prediction, beyond certain confidence intervals, indicates the 
relative effectiveness or ineffectiveness of the programs in the district. 

Plan for Developing Regression Equations ! 

. ^fultiple regression equations were developed for predicting each achieve- 
ment variable from the district input variables. Predictor variables were 
selected from among available measures with the intention that they represent 
the resources available to the district. The general procedure for develop- 
ing the prediction equations was to select and use a single set of predictor 
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variables for all dependent variables, but to calculate separate weights for 
these predictors for each of three sub-groups of districts. For each of the 
three district types: elementary, secondary, and unified, all 11 selected 
district characteristics were used to develop separate equations for pre- 
dicting the achievement test scores. 

Formation of Sub-groups 

Because the school districts in California are quite varied, it might be 
argued that they should not be put into the same group for the regression 
analysis. An extreme view is that each district is so unique that comparisons 
are not useful at all. The purpose of this study was to select a small num- 
ber of groups, which were reasonably homogeneous, through logical and to some 
degree empirical evidence. 

The breakdown of districts into unified, elementary, and high school was 
retained from last year's report. This categorization is widely used, and the 
equations developed on last year's data support the inference of differences. 
The educational processes involved in these groups are thought to be different. 
Therefore, it is reasonable to use different equations to predict outcomes. 

Four other measures were used to further subdivide the districts: ex- 
penditures per pupil, percent minority enrollment, index of family poverty, 
and total enrollment of the district. Aip. unified districts were divided 
into three groups of approximately equal numbers on the basis of each of the 
four measures. Multiple regression equations were then computed to predict 
sixth-grade reading scores and twelfth-grade reading scores for each sub- 
group. If the precision of prediction for one of these breakdowns was better 
than the precision for the unified districts as a group, such a breakdown 
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may be considered an improvement. It was found that only school district 
enrollment made an improvement, and further examination revealed that this 
difference was essentially eliminated when the prediction equations considered 
sample size. Therefore, the prediction equations were developed for the 
three types of districts without further subdivision. However, as described 
later, district size was used as a factor in estimating the precision of the 
predictions 

Selection of Predictor Variables 
0 

There is no single widely agreed upon method for deciding which measures 
should be included as predictors in regression equations (Draper and Smith, 
1966). In this study a number of criteria have been used for this selection. 
First, measures were included if they were found to be important predictors 
in the 1969-70 report. Secondly, measures which had wide interest (e.g., 
expenditures per pupil) were included. Thirdly, some other measures were 
included on the basis of some exploratory stepwise regression analyses. The 
11 predictors are listed below. 

1. Assessed valuation per unit of average daily attendance 

2. Percent minority pupils 

3. Percent Indian pupils 
k» Percent Black pupils 

5* Percent Oriental pupils 

6. Percent Spanish surnamed pupils 

7» Index of family poverty 

8. Pupil mobility 

9. Rate of staff tuniover '^-^^ ..^ 

10. Expenditures for instruction per unit of a.d.a. 

11. Regular a.d.a. for grades one through twe3.ve 

The input measures available are described in detail in Part I of the 
report of profiles. Tl^ measures studied were classified into three groups: 
(1) pupil characteristics; (2) school resources and characteristics; auid 
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(3) school allocation of resources. Only the measures in groups 1 and 2 
were included in the prediction equations. The reasons for including these 
and for excluding those in group 3 are given below. 



1. Pupil characteristic variables : 

9 Index of family poverty was included in the predictor set be- 
cause it was am importamt predictor (i.e., had a high correla- 
tion with test scores) in the 1969-70 data analysis. 

• Percent total minority was included in the predictor set be- 
cause it also was an important predictor in the 1969-70 data. 

• Percent Spanish surnamed, Black, Oriental, and American Indian 
pupils were not used in the 1969-70 report but were included 
here because they were expected to further refine the percent 
minority measure. 

• Pupil mobility index was included in the predictor set because 
it was a mildly important predictor in the 1969-70 study. 

2. School resources and characteristic veiriables : 

• Expenditures for instruction per pupil, though not a powerful 
predictor in the 1969-70 analysis, was included in the predictor 
set because it is commonly thought to be related to achievement 
and because it serves as a useful simmiary of available school 
resources. 

• Staff turnover was included as a predictor variable because it 
was found to be a mildly useful predictor in the 1969-70 data. 

• Assessed valuation per pupil was included because it is widely 
thought to be related to achievement. 
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• Size of school district (total regular a.d.a.) was included 
because it is commonly considered to be an important variable 
in comparing districts, although it actually added very little 
precision to the predictions. 

NOTE; Some measures were not included in the regression analyses 
for various reasons. Some were excluded because they were thought 
to reflect alternative allocation patterns or use of resources by 
the schools. The purpose of the regression analysis is to compensate, 
in part, for different input to the schools in terms of resources 
and student characteristics over which the schools have no control 
If some of these allocation measures were effective predictors 
of achievement, it would tend to increase the predicted scores for 
districts which used these factors extensively. Such districts 
would be unfairly penalized because of their resource allocation 
choices. The following variables were excluded for this reason. 

Resource allocation variables : 

• Minimum, maximum, and median teachers' salaries. 

• Average class size, grades 1-3 • 

• Pupil-teacher ratio, grades 4-8. 
o Percent non-teaching personnel. 

NQTEt Two other variables were not included in the equations be- 
cause their contribution to the precision of the equations was es- 
sentially zero. These were the two tax rate variables: total 
general fund tax rate and total general purpose tax rate. 

11 



Pupil scholastic ability scores (IQ) were also excluded, although they 
were included in the 1969-70 report and are very good predictors of achieve- 
ment test scores. There is very little evidence, if any, to indicate that 
ability test scores are subject to change due to a very higih or a very low 
quality instructional program as are achievement test scores. If this is 
true and if IQ scores were included in the equations, then districts with 
good programs would be penalized since the high ability scores of their 
pupils would lead to a high predicted score and, therefore, a depressed resi- 
dual. Conversely, districts with poor programs would be \anf airly rewarded. 

Development of the Equations 

Multiple linear regression equations were calculated with the use of 
BMD progi^'am 02R (stepwise regression) for all dependent variables for each 
type of district using the eleven predictor variables for each district. All 
calculations were based upon the predictors converted to normalized z-scores. 

Table 10 provides a summary of the multiple correlations obtained. All 
multiple correlations were statistically significant at alpha equal to .001. 
The multiple coefficients of determination (R2) are also given in Table 10. 
These indicate the proportions of variance in the predicted variable accounted 
for by the predictors. They range from .18 to .60, with a median of 
This indicates that about half of the variance among districts' scores is 
being predicted successfully. 

It csm be observed that at the elementary school level, the equations 
are more effective for the unified districts than the elementaiy districts, 
whereas at the secondary level the equations are more effective for the high 
school districts. It is also true that within the elementary level, the 
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accuracy of the equations is progressively better at higher grade levels • 
Cross Validation 

RegreesioTi equations tend to overestimate the amount of variance ac- 
counted for by the equations developed. This is due to the tendency to 
"take advantage" of chance relationships in the observed scores, relation- 
ships vhich are characteristic of the particular sample rather than the popu- 
lation which the sample represents* This tendency decreases as the number of 
observations increases. 

The best way to assess the validity of a regression equation is to use 
the equation developed on one sample of data to predict scores in a second 
sample of data. The effectiveness of these predictions is a measure of the 
validity of the regression equation. 

Another way of assessing the validity of regression equations is to use 
cross validation within the sample available. The sample is divided in half, 
prediction equations are developed independently for each hal^ and these equa- 
tions are then tried out on the opposite halves. If these equations tend to 
predict effectively for the half on which their validity is tested, we con- 
clude that some stable relationship is represented by the equations developed. 
This pj^ocedure was conducted for all achievement areas. 

Tables 11, 12, and 13 give the cross validation results for the three 
types of districts. In oach case, the total sample was divided into two 
groups on an odd-even basis, prediction equations were developed for each sub- 
sample, and these were used to predict scores in the other sub-sample. The 
tables present the proportion of variance accounted for by each equation for 
the total sample in the first column. The second column presents the average 
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r2 for the two halves when the weights were applied to the samples from which 
they were derived. The third column presents the average for the two 
halves when the weights were used to predict within the opposite halves. The 
difference between the latter two figures is also presented to aid the reader 
in judging the stability of thr equations. 

It can be seen that the equations for the elementary districts were 
most stable and those for high school districts were least stable. The loss 
in terms of was generally less than 5 percent for elementary districts 
but went as high as 33 percent for high school districts. The loss for unified 
districts varied from 5 to 20 percent. These trends are con'^istent with the 
logical expectation that the largest sample (elementary districts) would be 
the most stable and that the smallest sample (high school districts) would be 
the least stable. Generally, the results show a satisfactory level of sta- 
bility of the equations. 

Relative Contribution of Predictor Variables 

The question is frequently asked, "Which district characteristics are the 
most effective predictors of achievement?" 

Tables l4, 15, and l6 summarize the regression weights for each predictor 
for each of the sub-groups. Since the predictor variables are in approximately 
standard form (they were standardized across the total sample, though regres- 
sion weights were estimated separately for each type of district), the abso- 
lute value of eacl^l4gression weight reflects to some degree the contribution 
of a given predictor to the predictions of s criterion vsiriable. This rela- 
tionship is complex, however, due to the high intercorrelations among pre- 
dictor vsu^iables. In the data studied here, the predictor variables are so 
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highly intercorrelated that it is not possible to determine the unique con- 
tribution made by each predictor. If some variables had been omitted or if 
other variables had been added, the regression weights could have differed 
considerably. 

The proportion of variance accounted for is another measure frequently 
used in assessing the contribution of particular measures to the predictions, 
but it should be kept in mind that the proportion of variance accounted for 
is to some extent a reflection of the particular equations used (and therefore 
of the methods of selecting and ordering the variables used in those equations) 
and also of the intercorrelations among the selected predictors. 

Despite the limitations discussed above, it is useful to seek to compare 
the relative effectiveness of the predictor variables. For this purpose, 
each predictor measure was reink ordered in terms of its contribution to the 
explained variance (R^) using a stepwise regression procedoare (BMD02R; Dixon). 
These rankings, given in Tables 1? through 19i when compared across the pre- 
diction equations for a number of achie \t scores, give some idea of the 
relative effectiveness of predictors. Fc^- example, index of family poverty 
was usually the most effective predict of achievement for all three types 
of districts, while rate of staff turnov was one of the least effective. 

Within the context of the precautions mentioned above, the relative pre- 
dictive power of district characteristics may be summarized as follows: 

1. Assessed valuation per pupil tends to be a relatively poor predictor 
of achievement scores, although it increases slightly for the later 
grades in unified districts. 

2c Percent total minority pupils tends to be the most powerful predictor 
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for elementary districts, especially at the lower grades. Otherwise, 
it is quite weak, except for math in grade 12, 

5. The percentages of pupils in each of several ethnic groups were only 
sporadically good predictors of achievement, except for percent 
Oriental, which was a moderately good predictor for elementary and 
unified districts, and percent Spanish surnamed in unified districts. 

k. Index of family poverty was highly predictive of achievement test 
scores, especially for unified and high school districts. Overall, 
poverty is a more effective predictor of achievement than racial com- 
position of districts. 

5. Pupil mobility was a moderate to wesik predictor of achievement. 

6. Rate of staff turnover was a weak predictor. 

7. Expenditures for instruction was a weak predictor, except for the 
early grades where it was moderately effective. 

8. Regular average daily attendance was a relatively weak predictor. 

In svmmiary, index of family poverty was the most effective predictor, 
followed by the percent Oriental students and percent total minority pupils. 
It should also be said that the actual improvement to the precision of the 
equations made by the addition of all the predictors after the first two or 
three was generally quite small. 

Computation of Confidence Bands 

The development of regression equations, the estimation of regression 
weights, and the calculation of predicted scores were based upon observed scores 
transformed into normalized z-scores. Prediction equations were developed for 
each achievement measure for each of the three types of district organization. 



The districts of each type of organization (unified, elementary, and high 
school) were subdivided into ten size groupings. Means and standard deviations 
of the residuals were calculated for each of these sub-groups. As expected, 
the variance of the residuals was negatively correlated with district size, 
i.e«, the scores for large districts can be predicted more accurately than 
those for smaller districts. It was essential, therefore, that size of dis- 
trict be considered in determining the width of the confidence bands. Based 
upon an inspection of these variances, sub-groups with similar vsiriance were 
combined by collapsixig sub-groups similar In size and residual variance to 
form three to five larger sub-groups for each type of district. The vsu^i- 
ance of the residuals was then calculated for each of these new sub-groups. 

Setting a confidence level is somewhat arbitr^\ry. If it were to be set 
very high, only a few districts would appear to have achieved differently 
than expected. Setting it low would result in the exaggeration of slight 
differences between observed and predicted scores. The purpose of the con- 
fidence intervals was to supply a simple but meaningful guide to interpreting 
the achievement test results in the light of the resources available to the 
district. Fifty percent confidence intervals were selected as yielding an 
adequate but not overwhelming number of districts which actually achieved 
above or below the predicted score range. For 50 percent confidence intervals, 
about one- fourth of the districts fall above and about one- fourth fall below 
the predicted scores. 

For each of the sub-groups, the standard deviation of the residuals was 
multiplied by .67. Tables 20, 21, and 22 present the results of this operation. 
Fifty percent confidence intervals were calculated for each grade level and 
content area fpr every district by adding and subtracting these computed 
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values to the predicted scores. The resulting upper and lower boimdarieSi 
stated in terms of z-scores, were then converted to . percentile raiJcs. These 
percentile bands for each district can be found on Table 9 in Part II of the 
report, California State Testing Program , 1970-71 i Profiles of School District 
Performance t under the column headed, ^'Percentile Ranks of Predicted Score 
Range." The precision of the predictions, and therefore the width of the con- 
fidence intervals, will vary according to the size of the district and the 
effectiveness of the equation for that particular dependent variable. The 
calculation of the Performance Index ("A" for achievement above the predicted 
level; "B" for achievement below predicted level; and "W" for achievement 
within the expected range) was done as the last step, i.e., after the bands 
had been converted to percentile ranks. 

How to Calculate a Predicted Score Range 

To Tinderstand more clearly how the predicted B'^^re ranges were derived, 
the reader may wish to calculate one by hand. This can easily be accomplished 
by following the six steps outlined below. 

Step 1 . Convert the percentile ranks for the eleven predictors for a 

given district into unit normal deviates (z-scores). This can 
be done by consulting the z-tables which can be found in most 
standard statistical textbooks. For example, a percentile raiik 
of 16 would convert to a z-score of -1.0; a percentile rank of 
50 would convert to 0.0; Bk to +1.0; etc. 

Step 2 . Place these z-scores into the standard equation which reads 
as follows: 

y = C + (Xi) (bi) + (X2) (b2) + (X3) (b3) + . . . + CXii) (bii) 
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Where 



Step 3 > 



Step k . 



Step 5> 



Step 6 , 



Y = the predicted score in z-score terms 
C = the constant ■ 

X^.n = the z-score values for a district for each predictor 
variable (from Part I of California State Testing 
Program , 1970-71 ^ Profiles of School District 



The constants and the regression weights are to be found on 
Tables 10 to 12. These values 6a*e listed separately for each 
test score variable according to type of district. 
Complete the equation by multiplying each predictor variable 
z-score by the corresponding weight. Sum these products and 
add the constant to this total* This calculated value is the 
predicted score in z-score terms. 

By consulting Tables 20 to 22, locate the confidence band value 
appropriate for this test score vsofifiible for districts of this 
size and type. 

Compute the lower end of the predicted score range by sub- 
tracting the confidence band value from the predicted score 
obtained in Step 3. Find the upper le 1 of the predicted 
score remge by adding the confidence band value to the pre- 
dicted score. 

Convert the lower and upper values of the predicted score 
obtained in Step 5 (now in z-score terms) into percentile ranks 
by using the same tables mentioned in Step 1. 
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the partial regression weights for each predictor. 
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These calculations will result in a predicted score range for a given 
test variable for a given district which is the same or very similar to that 
found in Table 9 of the publication entitled, California State Testing Program 
1970-71 , Profiles of School District Performance > The two values may not be 
exactly the same since all values found on Table 9 were based on predictor 
values and regression weights carried to four decimal places. 

NON-LINEAR ANALYSIS 

The regression equations developed in this paper have been based upon 
linear relationships, that is, a unit of increase in a predictor variable 
is associated with a corresponding increase in the predicted variable. 

There are many other possible relationships, including non-linear re- 
lationships, such as a quadratic relationship in which high and low values 
of the independent variable are associated with low values in the dependent 
measure while medium values of the independent measure are associated with 
high values of the dependent measure. 

It may be hypothesized that such non-linear relationships exist in the 

I. 

present data and their use in the prediction equations would result in more 
effective prediction. A series of exploratory analyses were undertaken to 
examine this question. 

In one exploration, linear, quadratic, and cubic components of the 
predictor variables were examined as bivariate predictors of median sixth - 
grade reading scores. The results indicated no significant improvement in 
prediction by the use of the non-linear components. Correlations were 
usually only improved by .01 (e.g., .58 to .59) by the inclusion of quadratic 
and cubic components. 
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In another examination of this question, the quadratic components (in 
the form of the variables squared) were included with the original variables 

in multiple re^^sdxm^guay 

12th-grade reading scores. The increase in variance accounted for (R^) was 
3 percent for both grade levels; .56 to .39 for grade six; and .^5 to .^8 
for grade twelve. Such increments do not warrant the inclusion of quadratic 
components in the equa'*'i.ons. 

FACTOR ANALYSIS 

Considering the correlation matrices presented in Tables 1 to 9 and 
what is known about the measures involved, it seems likely that a simpler 
dimensional structure is possible. In this regard, some exploratory steps 
were taken to factor analyze the correlation matrices in order to find a 
simpler structure for the scores. Some conclusions based upon these analyses 
are summarized below. 

The factor analysis (principal component solution, number of factors 
equal to the number of eigenvalues greater than one, varimax orthogonal ro- 
ation, using the BMD03M program) of the predictor variables for elementstry 
districts (Table 5) yielded five factors. 

Factor one (expenditure): minimum salary, maximum salary, median 

salary, expenditure/a. d. a. 
Factor two (minority enrollment): percent minority, Spanish sumamed. 
Factor three (clase size): class size, pupil-teacher ratio. 
Factor four (tax rate): general fund tax rate, general purpose tax rate. 
Factor five (poverty) : non-teaching personnel, index of family poverty^ 

percent Black students. 
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For xmified districte, a similar analysis also yielded five factors. 

Factor one (expenditure-class size): expenditure per a.d.a,, regulstr 

a#d,a,, class size (negative), pupil-teacher ratio (negative). 
♦I Factor two (salaries): number non-teaching personnel, minimum teacher 

salary, maximum teacher salary, median teacher salary. 
Factor three (poverty-minority) : index of family poverty, percent 

total minority, percent Spanish eumamed pupils, percent Black 

pupils. 

Factor four (tax): general fund tax rate, general purpose tax rate. 
Factor five -(size): a.d.a., turnover (negative). 

These results suggest a five- dimensional structure of the predictors 
used. These five dimensions may be described as expenditure, percent minority, 
poverty, tax rate, and class size. 
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EPILOGUE 



It is to be understood that the technique of minimizing district dis- 
similarities by focusing on residuals between actual and predicted perfor- 
mance is still in the formative stage* Any comments or suggestions regarding 
the techniques and discussions described in this supplement or the book of 
profiles are most welcome* 

Future reports will be modified; as this one differs from previous re- 
ports. The input veu^iables will undoubtedly be changed, since AB 665 of 
1972 deleted the specific list of factors. New predictor variables will be 
added, especially some from the 1970 census. The new entry-level test to 
be introduced in grade one will be a prime predictor of reading scores at 
grades two and three. Indeed, that is its central purpose. A similar ap- 
proach that is being considered is the use* of achievement scores at one grade 
level as predictors of achievement at higher grade levels* However, further 
experimentation and consideration with this approach are needed before its 
possible adoption. 
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APPENDIX 



Table Number and Title 

1. Intercorrelation Matrix for the Eleven Test Score Variables 
for Unified Districts 

2. Intercorrelation Matrix for the 22 Predictor Variables for 
Unified School Districts 

3. Intercorrelation Matrix for Eleven Test Score Variables 
and 22 Predictor Variables for Unified School Districts 

k. Intercorrelation Matrix for Seven Test Score Variables for 
Elementary School Districts 

5* Intercorrelation Matrix for the 20 Predictor Variables for 
Elementary School Districts 

6. Intercorrelation Matrix for Seven Test Score Variables and 
20 Predictor Variables for Elementary School Districts 

?• Intercorrelation Matrix for Four Test Score Variables for 
High School Districts 

6. Intercorrelation Matrix for l8 Input Predictor Variables 
for High School Districts 

9. Intercorrelation Matrix for Four Test Score Variables and 
l8 Predictor Variables for High School Districts 

10. Summary of Multiple Kegreesion Analyses for Each Test 
Variable by Type of District 

11. Results of Cross-Validation Study, Unified Districts 

12. Results of Cross-Validation Study, Elionenteury Districts 

13. Results of Cross-Validation Study, High School Districts 

1^* Standard Score Regression Weights for Predicting Median 
Achievement Test Scores from Eleven School District 
Characteristics, Unified Districts 

15« Standard Score Regression Weights for Predicting Median 
Achievement Test Scores from Eleven School District 
Characteristics, Elementary School Districts 

l6. Standard Score Regression Weights for Predicting Median 
Achievement Test Scores from Eleven School District 
Characteristics, High School Districts 



2k 



I 



Table Number and Title Page 

17. Ranks of Predictor Variables in Terms of Order of Entry lf2 
in Stepwise Regression Program, Unified Districts 

18. Ranks of Predictor Variables in Terms of Older of Entry k3 
in Stepwise Regression Program, Elementary Districts 

19. Ranks of Predictor Variables in Terms of Order of Entry kk 
in Stepwise Regression Program, High School Dicstricts 

20. Probable Error of Estimate in Predicting Median Achievement k5 
Test Scores by Average Daily Attendance of District, Unified 
Districts 

21. Probable Error of Estimate in Predicting Median Achievement kS 
Test Scores by Average Daily Attendance of District, 
Elementary Districts 

22. Probable Error of Estimate in Predicting Median Achievement k7 
Test Scores by Average Daily Attendance of District, High 

School Districts 
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Table 11 



Results of Cross-Validation Study 
Unified Districts 



Proportion of Variance Accounted for (R^) 



Dependent 
Variable 


Total 
Sample 


(A) _ 
Avei'age R'^ 
for Cdd-Even 
Subsamples 


n2 

Average R'^ 
from Cross- 
Validation 


Difference 
(A-B) 


1 Reading 


.3262 


.3623 


.2095 


.1528 


2 Reading 


.^615 


A797 


.'^199 


.0598 


3 Reading 


.5609 


.6000 


A73k 


•12*fo 


6 Reading 


.5559 


.6051 


A397 




6 Language 


.5895 


.6015 


.525^ 


.0761 


6 Spelling 


Ask? 


.5358 


.3363 


.1995 


6 M&thematics 


.5636 


.5880 


Ak67 


.1^13 


12 Reading 


Ak62 


.if 832 


.3153 


.1679 


12 Language 


A303 


.kSlS 


.5512 


.llOif 


12 Spelling 


.3661 


.3837 


.3101 


.0736 


12 Mathematics 


.4965 


.^932 


,mi 


,0k31 
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Table 12 



Results of Croas-Validation Study 
Elementary Districts 



Proportion of Variance Accounted for (R ) 



Dependent 
Variable 


Total 
Sample 


(A) - 
Average R^ 

for Odd-Even 

Subsamples 


(B) 5 
Average R'=^ 

from Cross- 

Validation 


Difference 
(A-B) 


1 Reading 


.1801 


.1859 


.1615 


.02lflf 


2 Reading 


.2550 


.27^2 


.2093 


.06if9 


3 Reading 


.33^3 


.3690 


.3071 


.0619 


6 Reading 




.39H 


.3772 


.0172 


6 Language 


.2891 


.320k 


.2838 


.0366 


6 Spelling 


.2589 


.2889 


.2631 


.0258 


6 Mathematics 


'255k 


.2879 


.2608 


.0271 
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Table 13 



Results of Cross-Validation Study 
High School Districts 



Proportion of Variance Accounted for (R ) 



Dependent 
Variable 


Total 
Sample 


00 

Average 
for Odd-Even 
Subsamples 


CBD ' 

Average R 
from Cross- 
Validation 


Difference 
(A-B) 


12 Reading 


.6005 


.66if7 




.2351 


12 Language 


.5086 




.hl.55 


.1209 


12 Spelling 


.369^ 




.1387 


.3308 


12 Mathematics 


.5822 


.6168 


.^^705 


.lif63 
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Table 20 



Probable Error of Estimate in Predicting Median Achievement Test Scores 
by Average Dally Attendance of District 

Unified District 



Output Variable 
by Grade 


Error of Estimate by ADA Range 


0-462 


463-2370 


2371-7434 


7435+ 


4k 

1 

2 
3 


...0.6188 
0.6256 
0.5644 


0.4420 
0.3876 
0.3536 


0.3332 
0.3604 
0.2924 


0.2652 
0.2380 
0.2380 



Output Variable 
by Grade 



Error of Estimate by ADA Range 



0-462 



463-2370 



2371-4242 



4243+ 



6 Reading 
6 Language 
6 Spelling 

12 Reading 
12 Language 
12 Spelling 



0.5916 
0.5100 
0.5372 

0.7752 
0.7684 
0.8228 



0.4216 
0.4216 
0.4284 

0.5168 
0.4420 
0.5644 



0.2856 
0.3060 
0.2788 

0.4352 
0.5440 
0.5916 



0.2652 
0.2788 
0.2856 

0.4148 
0.4216 
0.4216 



Output Variable 


Error of Estimate by ADA Range 


by Grade 


0-2370 


2371-7433 


7434-16028 


16029+ 


6 Mathematics 


0.4760 


0.3332 


0.2516 


0.2924 


12 Mathematics 


0.8160 


0.4012 


0.4692 


0.3468 



^^5 



Table 21 

Probable E|rror of Estimate in Predicting Median Achievement Test Scores 



) I 



■t. i 



by Average Daily Attendance of District 
Elementary Districts 



Output Variable 



Error of Estimate by ADA Range 



by Grade 


6-37 


38-80 


81-208 


209-492 


493-1290 


1291+ 


1 


0.972^ 


0.7684 


0.7888 


0.6800 


0.4896 


0.3332 


2 


0.9792 


0.7616 


0.7480 


0.5848 


0.3876 


0.3060 


3 


0.9112 


0.7752 


0.6732 


0.5168 


0.4896 


0.2856 


6 Reading 


1.0132 


0.7072 


0.6392 


0.4420 


0.3604 


0.2720 


6 Language 


0.9384 


0.8024 


0.6664 


0.4760 


0.3740 


0.3264 


6 Spelling 


1.0472 


0.8364 


0.6936 


0.5100 


0.4012 


0.3060 


6 Mathematics 


1.0132 


0.8704 


0.6324 


0.5100 


0.3468 


0.3332 



Table 22 



Probable Error of Estimate in Predicting Median Achievement Test Scores 
by Average Daily Attendance of District 

High School Districts 



Output Variable 


Error of Estimate by ADA Range 


by Grade 


0-32k 


525-185^ 


1855-8770 


8771-21957 


21958+ 


12 Spelling 


0.700^t 


0.^+896 


0.4080 


. 0.3332 


0.19j4 



Output Variable 


Error of Estimate by ADA Range 


by Grade 


0-524 


525-3525 


3526-8771 


8772+ 


12 Reading 
12 Language 


0.5440 
0.6120 


0.3468 
0.3604 


0,4624 
0.4760 


0.1972 
0.1768 



Output Variable 


Error of Estimate by ADA Range 


by Grade 








0-1854 


1855-12448 


12449+ 


12 Mathematics 


0.4488 


0.3604 


0.1224 
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